When states of right front, sides, back, or the like of a vehicle in a traveling direction are not directly visible from a driver's seat, there is a danger that the vehicle may crash into a hidden obstacle which is located in the traveling direction. A system for avoiding such a danger includes a driving assistance system that synthesizes images captured by a plurality of wide-angle cameras mounted on the front, back, and lateral sides of a vehicle, generates a virtual image observed from above the vehicle, and displays the image on a display unit provided at a driver's seat.
Meanwhile, the image observed from above the vehicle is referred to as an “around-view image”, an “overhead image”, a “birds-eye image”, or the like. Hereinafter, in this specification, a description will be given on the assumption that the image observed from above the vehicle is an “overhead image”.
In such a system, the camera installed in the vicinity of the vehicle is desired to capture an image having a region as wide as possible at close range, and thus includes a wide-angle lens such as, for example, a fish-eye lens. The camera can capture an image having a wide region by using the wide-angle lens. However, image distortion occurs in the image captured using the wide-angle lens. In particular, there is a problem in that a large amount of distortion occurs in a region in the vicinity of the image.
A general driving assistance system of the related art is configured to execute distortion correction of a captured image, to generate an “overhead image” viewed from above a vehicle as a corrected image, and to output the generated image to a display unit.
Meanwhile, an image correction process for generating the overhead image is referred to as “overhead conversion”.
However, positions at which cameras are mounted on a vehicle include a front, a rear, sides, and the like of the vehicle. A range in which each camera is capable of capturing an image is limited to any one region of front, back, right, and left regions of the vehicle. Accordingly, in order to generate an overhead image including an all-around image of the vehicle, it is necessary to execute a synthesis process of joining the overhead images which are generated on the basis of images captured by a plurality of cameras located at different positions.
However, when the synthesis process is performed of joining a plurality of overhead images which are generated by performing overhead conversion on images captured using a wide-angle lens, subjects in junction regions between the plurality of images may be erased without remaining in a synthesis image. This is because distortion correction is performed using the ground as a reference plane. For example, when there is a subject (stereoscopic object) which is located at a higher position than the reference plane, the subject is pressed out of a boundary of a junction between the images after overhead conversion, and thus may not remain in the synthesis image.
For example, it is assumed that a stereoscopic object X extending upwards from the ground, which is equivalent to the reference plane in the overhead conversion, is captured in the vicinity of substantially a boundary of each of two of an image A and an image B which are objects to be synthesized.
When a synthesis image is generated by joining corrected images (overhead images) obtained by performing overhead conversion on the image A and the image B, a junction region between the image A and the image B is subjected to a joining process in which a correspondence position of the ground which is the reference plane of the overhead conversion is set to a junction line. The process of joining the images A and B is performed by cutting off an image outside the junction line between the images A and B.
When such a joining process is performed, a situation may occur where the stereoscopic object X captured in the image A is set to the outside of the junction line of the image A, that is, the region cut off from the image A. On the other hand, the stereoscopic object X captured in the image B is also set to the outside of the junction line of the image B, that is, the region cut off from the image B.
As a result, the stereoscopic object X in the vicinity of the boundary between the images A and B does not remain in the synthesis image. That is, it is not possible to confirm if the stereoscopic object X, which is an originally-existing subject, is in the synthesis image.
A technique of the related art for solving such a problem is PTL 1 (Japanese Unexamined Patent Application Publication No. 2007-109166). PTL 1 discloses a configuration in which images captured by imaging devices mounted on the periphery of a vehicle are synthesized by overhead conversion so that a synthesis image viewed from above the vehicle is generated and displayed. PTL 1 discloses a configuration in which two corrected images are alternately displayed in a comb shape in a region where two overhead images as objects to be synthesized overlap each other.
A subject pressed out of the image by overhead conversion accompanied by distortion correction is displayed in the comb-shaped display region. Thus, a driver who is an image observer can confirm that some kind of object is present in the region, and can pay attention to the object.
PTL 1 discloses a configuration in which a region having two overhead images overlapping each other is displayed in a fixed pattern in which the two overhead images are alternately set in a comb shape. However, in overhead conversion using a ground plane as a reference plane, the conversion is not correctly performed on a stereoscopic object displayed in the comb-shaped display region. Thus, distortion remains, and the original shape is not restored. In addition, since the display is disconnected due to the two images being displayed in a comb shape, it may be difficult to cause an observer (driver) to recognize the presence of a stereoscopic object having a possibility of collision.